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ABSTRACT 

The National Assessment of Educational Progress's 
efforts to measure progress toward a commonly accepted educational 
goal - vocational satisfaction and competence - are discussed* The 
attempt to operationalize the goal resulted in a set of capabilities 
prerequisite to the ultimate goal attainment* The difficulties with 
this "general" set are considered, one of which is that it does not 
assess capabilities specific to any given occupation. Such a task is 
viewed as overwhelming, necessary, but not susceptible to immediate 
solution* The remainder of the paper elaborates some possible interim 
assessment procedures. The author concludes that the assessment of 
vocational education to date has been ambitious and rigorous and that 
the difficulties inherent in the process are now exposed* (TL) 
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u S DEPAATMtflT OF HEALTH. EDUCATION 

A WfLFAHt 

CFFlCt OF EDUCATION 
THIS DOCUMENT HAS BEEN REPRODUCED 
exactly as received from the person or 
ORCANIZATiCN originating it points of 
VIEW DR OPINIONS STA’ED DO NOT NECES 
SAFILY REPRESENT OFFICIAL OFFICE OFEDU 
CATION POSITION OR POLICY 

Institutions, like me sc organized social activities, are established fnr 
reasons. Tha- 3 , 1 are set up to achieve goals or purposes. These goals 

are .important in our present discussion because any assessment of progress 
requires knowledge of the destination or goals toward which progress is intended, 
of the intended routes to the goals, and of any alternative, acceptable routes. 

Further, any responsible progress assessment must report on progress toward the 
announced goals of the institution, \*hatever other interesting or important 
things it might reveal. 

A substantial number of individuals and groups have attempted U> put into 
writing the goals of public education. Grgne (3965) reports that his analysis 
of these attempts reveals three goals for students on which there is at least 
verbal consensus: responsible citizenship, vocational satisfaction and competence, 

and the capacity for participating in and sharing a variety of aesthetic experiences. 

My experience with the statements of goals or purposes of some schools and school 
districts supports the hunch that many individual, schools and district: hold these 
three purposes and vould agree that they are their proper destination. 

A basic task of an educational assessment effort, then, would seem to be to 
provide evidence concerning progress tov\rd these three goal 3 for students. 

Whether the National Assessment of Educational Progress did, in fact, set out 
self-consciously to measure progress toward these three goals, I do not know. 

However, they did bat at least .667 because two of the areas designated for study 
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very early in their efforts were citizenship and vocational education. It is 
important that we remember during this discussion that vocational satisfaction 
and competence is widely rcognized as a goal of education for all students and 
that progress toward this goal was what we intended to assess. 

However useful such a goal as 'Vocational satisfaction and competence " 
might be for same purposes, it is nearly worthless in that form as a guide to 
assessment of progress. Somehow, such a goal must be translated into something 
which can be verified by observation. Further, if progress is to be assessed 
through observation of student performance, it is important to know the 
prerequisites to achievement of the goal. That is, it is important to know 
what must be learned en route to the goal. 

The Assessment staff and its contractors vent about the task of redefining 
the goal and identifying intermediate objectives quite systematically. They 
began by asking the question, "What would a person have to know or be able to 
do if he is to be vocationally satisfied and competent? " The answer finally 
selected was that he must be capable of career decision-making (a life-long 
process), of developing or forging a career, and of performing the tasks required 
by an occupation. Each of these prerequisites to vocational satisfaction and 
competence then vac analyzed in a similar way to produce its prerequisites and 
the process was continued until statements were reached which described relatively 
simple capabilities. This procedure resulted in a set of capabilities (educational 
objectives) arranged in an initial hierarchical structure. It might be noted 
that capabilities toward the bottom of this structure tended to be not only 
relatively simple, but also general and recognizable as from such other Assessment 
areas as Heading and Writir^, Mathematics, and Science. 

The entire set had several very important advantages. It provided an 
unusually goocl definition of what is meant by 'Vocational satisfaction and 
competence." It provided a clear rationale and an explicit basis for specifying 
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exercises in the assessment. The structure identified dependencies between 
capabilities and, thus, provided a basis for selecting particular capabilities 
for assessment at the several age levels. Distance from the goal vas definable, 
even for individual students, in terms of achievement position in the sequence 
of capabilities. Finally, a great variety of capabilities vas included in the 
structure, including many vhich vere non-technical or non-job-specific. 

On the other hand, same significant problems vere exposed as the analysis 
displayed the considerable complexity of the goal. For example, the structure 
vhich developed vas a partially-ordered set. That is, vertical relations betveen 
capabilities vere defined, but horizontal relations vere not. One immediately 
obvious result vas that a capability might be (and vas) found in several branches 
of the structure. There vas, in other vords, some overlap of parts of the 
structure. This vas a troublesome, but velcome, result since such recinrring 
capabilities indicate a kind of generalizability. A second and less velcome 
problem vas (and is) that some capabilities identified in the analysis can be 
learned only after their prerequisites have bee r acquired and are not equal to 
the sum of their prerequisites. Even if the student vere capable of ell of the 
prerequisites to such an higher-order capability, he might not be capable of 
integratirg the prerequisites to achieve the higher-order capability. This 
raised the serious question of the proper level at vhich to vrite assessment 
exercises. But the most difficult problem, and one vhich still is vith us, 
vas presented by the sheer number of capabilities to be assessed. Hundreds of 
capabilities vere defined by the analysis, far more than could be covered by 
the assessment. Hov could they be sampled to permit a sensible report? 

The largo number of capabilities available for assessment vas considered 
an especially serious problem in that branch of the structure addressed to 
"perform the tasks required by an occupation. 11 The problem here vas not different 

formally from that in any other part of the analysis, but the tremendous number 

a 

f occupations include an extremely large number of tasks, so many that even 
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enumerating them seems unthinkable and any attempt to assess their performance 
c^pletely impractical. This presented the Assessment vith a dilemma. It vould 
be able to assess many peripheral, preparatory, facilitating, and important 
capabilities, but it vould not be able to make comprehensive assessment of those 
capabilities for vhich people are hired, the so-called marketable skills. 

This predicament vas regretable, of course, but the Assessment did about 
the only thing it could do under the circumstances: it decided not to try to 

assess "specific" vocational capabilities, in the first cycle at least, and 
renamed the area "Career and Occupational Development" vhich vas thought to 
describe more accurately vhat ves being assessed. One prominent vocational 
educator still argues that it should have been named "Career and General Occupa- 
tional Development," or more pointedly, "Career Planning and Development," to 
make it as clear as possible that the Assessment is addressed only to those 
components of vocational preparation common to many or all occupations and 
now is making no attempt to determine the extent to vhich the schools are 
preparing students vith capabilities for vhich they can be employed. 

This turn of events saddened vocational educators and some others vho 
realised that a major goal of education, one to vhich they are ievoting much 
of their effort and tovard v?iich several million students arc striving, vould 
not be included in a National Assessment vhich they strongly support. They 
agree that present C.O.D. objectives are important and should be assessed - 
in fact, they helped identify them - but they also believe vocational competence 
to be ft goal of consequence for all students vhich r-ust be assessed and they 
are not villing yet to admit defeat. 

It ia true that individual vocational educators, includirg me, have specific 
disagreements vith the decisions about seme particular items in the present set, 
about some measurement matters, about some porta of the rationale presented, 
even about aspects of the overall Assessment strategy. The Assessment staff 



can testify to the persistence with which some of as have pressed our points. 

But the same can be said about most groups participating in the development work. 
These are relatively minor problems which cannot deter vocational people from 
their efforts to get vocational education assessad. These people have endured 
decades of lip service, inadequate funding and poor publicity in spite of lofty 
pronouncements about the importance of vocational satisfaction and competence. 

They do not want to see this state if affairs perpetuated and given indirect 
sanction even unintentlorially by exclusion of vocational assessment from this 
major educational event. Nor do they want the Assessment’s emission of vocational 
content to be interpreted i\s abandonment of vocational preparation as a goal of 
public education# It is such concerns as these, and an eternal optimism in the 
face of sustained adv^sity, which requires vocational educators to press for 
eventual inclusion of vocatiotal objectives in the assessment. 

Since it usually is much easier to identify a problem then to solve it, 
and since a solution to this problem may be some time in development, perhaps we 
should consider briefly seme constructive things which might be done while 
vaiirg for the Assessment doctor to arrive. 

One possibility, which may go outside the usual procedure in this Assessment, 
would be to make us'* of indirect evidence concerning the acquisition of marketable 
capabilities. To some extent, this method is being use in C.O.D. for assessment 
of career implementation skills# It could be extended to include evidence from 
activities after school, duri»^ summers, even in school, as to the acquisition 
of skills. A fertile imagination should turn up numerous kinds and sources of 
information justifying some conclusions about how ready students or graduates 
are for vocational responsibility, 

A second, possibility, consistent with the C#0 #D# framework, would be to 
expand the present coverage of "gfnerally useful skills" to provide more 
systematically for the assessment of kinds of capabilities required in a wide 
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variety of occupations. Altman's (l$)66) formulation, based on empirical studies, 
provides a set of defensible content areas which he combines with a useful set 
of psychological processes to define a process x content matrix within which 
many marketable capabilities of general usefulness can be identified. 

I would not want this suggestion to imply undue criticism of the exercises 
planned for this part of the C.O.D. assessment. They are, I think, among the 
best. I do think that if the objectives and their related exercises were sorted 
into Altman's content x process categories we would find that a substantial 
number of categories have no representation in the assessment, that some important 
category groups are among the omissions, and that some improvements could be 
wrought in the distribution of exercises among categories, their work -relatedness, 
and their selection for explication of age-group differences. 

In some ways, the simplest maneuver would be to go ahead and sample 
capabilities from the huge area of marketable skills, makir^ the best possible 
assessment under the limitations which exist on time, money and subjects. It 
should be possible to identify a number of kinds of occupations which are among 
the most populous and, from these, to selecb a number of tasks which are widely 
required and reasonab]/ measurable. Perhaps, a pool of such exercises could be 
started and increased with each assessment cycle. The effort could be made a 
bit more sophisticated if, for the selected test tasks, the prerequisite 
capabilities were identified and used in sequential fashion to assess progress 
at the several age levels or stages of progress. Some time might be saved in 
the administration of such exercises if some kind of sequential procedure were 
used, as suggested by Bohn, to identify in advance persons unable to do time- 
consuming exercises. In any case, it would bo possible to report on the 
proportion of the population capable of a variety of significant, marketable 
kinds of performances, and it seems to me much better to report that much than 
to say nothing about an area of such considerable importance. To those who 
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feel squeamish about the inevitably miniscule proportion cf tasks vhich can be 
sampled, I vould argue that every other area of assessment vould be subject to 
the same criticism if they had defined and analyzed an end-product goal vith 
comparable rigor* 

These suggestions are stop-gap measures, of course, though they are not 
clearly less effective than procedures applied in other areas of the Assessment. 

In the end, it vould seem, the entire Assessment vould benefit if it could be 
made from a more comprehensive and durable conceptual base than an inventory of 
current task requirements. In the area of vocational satisfaction and competence, 
it is important to remember that jobs are collections of tasks defined by a 
number of transient considerations. They can be changed and redesigned quite 
readily and, thus, provide an unstable basis for planning or assessing educational 
programs. But every task, regardless of economic value, can be vioved as a 
specific example of the application of sane kinds of human capacities to sane 
kinds of content. If ve could describe the possible varieties of human 
activity by reference to domains vitbin vhich generalization of learning occurs, 
ve vould be able to sample performance vithin a domain aixi estimate vith some 
confidence the probability of successful performance on any particular activity 
in tbs domain, 

Obviously, this is a long-term, difficult u.. a staking, But help is on the 
vay. Altman's ( 1966 ) imaginative study of generalizable vocational capabilities, 
Fleishman's (1970 ) vork on a taxonomy of human performance, Gagne's recent 
report to The American Educational Research Association on domains of generalization 
about learning, and the ser e ral scattered studies attempting to identify clusters 
of capabilities and jobs, all are related to the problem. 

In closing, let me restate a point made earlier, I do not think vocational 
assessment is in voree shape than the other areas of assessment. I do think ve 
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have tackled a much larger, more ambitious goal than most other arecs and I think 
that we have analyzed that goal with unusual rigor. We have demanded more 
justification for each item of the assessment in vocational education than in 
any other. As a consequence, we know better than most what is required for an 
adequate assessment and we have exposed to full view the difficulties which would 
he found in any area to which a comparable process were applied. 
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